A portion of the disclosure of this patent document contains material which is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent disclosure, at it appears in the Patent and Trademark Office files or records, but otherwise reserves all copyright rights whatsoever.
Background of the Invention
Summary of the invention
Brief Description of the Drawings
Detailed Description of the Preferred Embodiment
Part 1.
System Overview
Part 2.
Three-Dimensional Image Capture
Scanner Manufacture and Calibration
Pattern Recognition
Decoding
Derivation of 3-D Point Cloud per Image
Part 3.
Generation of Digital Impression
Entry point to registration
Frame to Frame Registration
Cumulative Registration of Entire Jaw
Segment registration
Landmarking
Separation of Teeth into Individual Tooth Objects (tooth modeling)
Part 4.
Treatment Planning
Part 5.
Appliance Manufacturing
Robot Design
Archwire Manufacture
Claims
Abstract
A. Field of the Invention
This invention relates generally to the field of orthodontics. More particularly, the invention relates to a computerized, interactive method and associated system for orthodontic treatment. The system includes a hand-held optical scanner capturing 3-dimensional information of objects, interactive computer-based treatment planning using three-dimensional tooth objects and user specified simulation of tooth movement, and appliance manufacturing apparatus, including bending machines.
B. Description of Related Art
In orthodontics, a patient suffering from a malocclusion is typically treated by bonding brackets to the surface of the patient""s teeth. The brackets have slots for receiving an archwire. The bracket-archwire interaction governs forces applied to the teeth and defines the desired direction of tooth movement. Typically, the bends in the wire are made manually by the orthodontist. During the course of treatment, the movement of the teeth is monitored. Corrections to the bracket position and/or wire shape are made manually by the orthodontist.
The key to efficiency in treatment and maximum quality in results is a realistic simulation of the treatment process. Today""s orthodontists have the possibility of taking plaster models of the upper and lower jaw, cutting the model into single tooth models and sticking these tooth models into a wax bed, lining them up in the desired position, the so-called set-up. This approach allows for reaching a perfect occlusion without any guessing. The next step is to bond a bracket at every tooth model. This would tell the orthodontist the geometry of the wire to run through the bracket slots to receive exactly this result. The next step involves the transfer of the bracket position to the original malocclusion model. To make sure that the brackets will be bonded at exactly this position at the real patient""s teeth, small templates for every tooth would have to be fabricated that fit over the bracket and a relevant part of the tooth and allow for reliable placement of the bracket on the patient""s teeth. To increase efficiency of the bonding process, another option would be to place each single bracket onto a model of the malocclusion and then fabricate one single transfer tray per jaw that covers all brackets and relevant portions of every tooth. Using such a transfer tray guarantees a very quick and yet precise bonding using indirect bonding.
However, it is obvious that such an approach requires an extreme amount of time and labor and thus is too costly, and this is the reason why it is not practiced widely. The normal orthodontist does not fabricate set-ups; he places the brackets directly on the patient""s teeth to the best of his knowledge, uses an off-the-shelf wire and hopes for the best. There is no way to confirm whether the brackets are placed correctly; and misplacement of the bracket will change the direction and/or magnitude of the forces imparted on the teeth. While at the beginning of treatment things generally run well as all teeth start to move at least into the right direction, at the end of treatment a lot of time is lost by adaptations and corrections required due to the fact that the end result has not been properly planned at any point of time. For the orthodontist this is still preferable over the lab process described above, as the efforts for the lab process would still exceed the efforts that he has to put in during treatment. And the patient has no choice and does not know that treatment time could be significantly reduced if proper planning was done.
U.S. Pat. No. 5,431,562 to Andreiko et al. describes a computerized, appliance-driven approach to orthodontics. In this method, first certain shape information of teeth is acquired. A uniplanar target archform is calculated from the shape information. The shape of customized bracket slots, the bracket base, and the shape of an orthodontic archwire, are calculated in accordance with a mathematically-derived target archform. The goal of the Andreiko et al. method is to give more predictability, standardization, and certainty to orthodontics by replacing the human element in orthodontic appliance design with a deterministic, mathematical computation of a target archform and appliance design. Hence the ""562 patent teaches away from an interactive, computer-based system in which the orthodontist remains fully involved in patient diagnosis, appliance design, and treatment planning and monitoring.
More recently, in the late 1990""s Align Technologies began offering transparent, removable aligning devices as a new treatment modality in orthodontics. In this system, a plaster model of the dentition of the patent is obtained by the orthodontist and shipped to a remote appliance manufacturing center, where it is scanned with a laser. A computer model of the dentition in a target situation is generated at the appliance manufacturing center and made available for viewing to the orthodontist over the Internet. The orthodontist indicates changes they wish to make to individual tooth positions. Later, another virtual model is provided over the Internet and the orthodontist reviews the revised model, and indicates any further changes. After several such iterations, the target situation is agreed upon. A series of removable aligning devices or shells are manufactured and delivered to the orthodontist. The shells, in theory, will move the patient""s teeth to the desired or target position.
The art has lacked an effective, computer-based interactive orthodontic treatment planning system that provides the necessary tools to allow the orthodontist to quickly and efficiently design a treatment plan for a patient. The art has also lacked a treatment planning system in which the orthodontist-derived parameters for the treatment can be translated into a design of an orthodontic appliance in real time, while the patient is in the chair. Real-time appliance design as described herein also allows for real-time communication of the treatment plan or appliance design to occur with the patient, or transmitted over a communications link and shared with a colleague or remote appliance manufacturing facility. Alternatively, the treatment planning can be performed remotely and a digital treatment plan sent to the orthodontist for review, interactive modification, or approval.
Scanners are devices for capturing and recording information from a surface of an object. Scanners for obtaining information from a two-dimensional surface, such as reading bar codes or characters printed on a piece of paper, are widely known. Several scanners have been proposed for recording three-dimensional information as well, including the field of dentistry.
U.S. Pat. No. 4,837,732 and U.S. Pat. No. 4,575,805 to Brandestini and Moermann propose a scanning system for in vivo, non-contact scanning of teeth. The patents describe a procedure for optically mapping a prepared tooth with a non-contact scan-head. The scan-head delivers the contour data, converted to electrical format, to be stored in a memory. A computer reads the memory following a line scan pattern. A milling device is slaved to follow this pattern by means of position control signals and mills an implant for the prepared tooth cavity.
The scan-head of the ""732 and ""805 patents includes a light emitting diode, with integral lens that radiates light onto the cavity. Before reaching the object, the rays of light are reflected by a mirror and pass through a ruling consisting of a plurality of parallel slits, or an alternating pattern of parallel opaque and transparent stripes. The reflected light is focused by a lens onto a charge-coupled device (CCD) sensor. Depth information is determined in accordance with a principle known as xe2x80x9cactive triangulation,xe2x80x9d using parameters shown in FIG. 9 of this document and described subsequently. Basically, the object is viewed under an angle different from the incident rays due to a parallax effect. Each light stripe will have an apparent positional shift and the amount of the shift at each point along each light stripe is proportional to the vertical height of the corresponding portion of the surface on the object.
U.S. Pat. No. 5,372,502 to Massen et al. describes an optical probe for measuring teeth that works on a similar principle. As noted in the Massen et al. patent, the Brandestini et al. technique is difficult to use when there are large variations in surface topography since such large jumps displace the pattern by an amount larger than the phase constant of the pattern, making it difficult to reconstruct the pattern of lines. Furthermore, precise knowledge of the angle of incidence and angle of reflection, and the separation distance between the light source and the detector, are needed to make accurate determinations of depth. Furthermore, the scanner has to be rather carefully positioned with respect to the tooth and would be unable to make a complete model of the dentition.
U.S. Pat. No. 5,027,281 to Rekow et al. describes a scanning method using a three axis positioning head with a laser source and detector, a rotational stage and a computer controller. The computer controller positions both the rotational stage and the positioning head. An object is placed on the rotational stage and the laser beam reflects from it. The reflected laser beam is used to measure the distance between the object and the laser source. X and Y coordinates are obtained by movement of the rotational stage or the positioning head. A three-dimensional virtual model of the object is created from the laser scanning. The ""281 patent describes using this scanning method for scanning a plaster model of teeth for purposes of acquiring shape of the teeth to form a dental prosthesis. The system of the ""281 patent is not particularly flexible, since it requires the object to be placed on the rotational stage and precise control of the relative position of the object and the positioning head is required at all times. It is unsuited for in vivo scanning of the teeth.
U.S. Pat. No. 5,431,562 to Andreiko et al. describes a method of acquiring certain shape information of teeth from a plaster model of the teeth. The plaster model is placed on a table and a picture is taken of the teeth using a video camera positioned a known distance away from the model, looking directly down on the model. The image is displayed on an input computer and a positioning grid is placed over the image of the teeth. The operator manually inputs X and Y coordinate information of selected points on the teeth, such as the mesial and distal contact points of the teeth. An alternative embodiment is described in which a laser directs a laser beam onto a model of the teeth and the reflected beam is detected by a sensor. The patent asserts that three-dimensional information as to teeth can be acquired from this technique but does not explain how it would be done. Neither of the techniques of Andreiko have met with widespread commercial success or acceptance in orthodontics. Neither technique achieves in vivo scanning of teeth. Moreover, the video technique does not produce complete three-dimensional information as to the teeth, but rather a limited amount of two-dimensional information, requiring significant manual operator input. Even using this technique, additional equipment is required even to describe the labial surface of a tooth along a single plane.
The art has also lacked a reliable, accurate, low-cost, and easily used scanning system that can quickly and automatically acquire three-dimensional information of an object, without requiring substantial operator input, and in particular one that can be held in the hand and used for in vivo scanning or scanning a model. The present invention meets this need.
An interactive, orthodontic care system is provided based on scanning of teeth. The system includes a hand-held scanner and associated processing system for capturing images of the dentition of the patient and processing the images to generate a full, virtual, three-dimensional model of the dentition. A data conditioning system processes the virtual, three-dimensional model and responsively generates a set of individual, virtual three-dimensional tooth objects representing teeth in the dentition of the patient A workstation is provided having a user interface for display of the set of individual, virtual three-dimensional tooth objects. Interactive treatment planning software is provided on the workstation permitting a user to manipulate the virtual three-dimensional tooth objects to thereby design a target situation for the patient in three dimensions and parameters for a customized orthodontic appliance for the teeth.
The hand-held scanner and associated processing system, data conditioning system, and workstation may all be installed in an orthodontic clinic. Alternatively, the hand-held scanner and workstation are installed in an orthodontic clinic, and wherein the data conditioning system is installed in a general purpose computer at a remote location from orthodontic clinic. Further, the treatment planning software can be either installed at the clinic, and/or at a remote location, and/or at a precision appliance manufacturing center that manufactures a custom orthodontic appliance. The type of appliance may vary considerably.
The treatment planning apparatus can be considered an interactive, computer-based computer aided design and computer aided manufacturing (CAD/CAM) system for orthodontics. The apparatus is highly interactive, in that it provides the orthodontist with the opportunity to both observe and analyze the current stage of the patient""s condition and develop and specify a target or desired stage. A shortest direct path of tooth movement to the target stage can also be determined. Further, the apparatus provides for simulation of tooth movement between current and target stages.
In its broader aspects, the apparatus comprises a workstation having a processing unit and a display, and a memory storing a virtual, complete three-dimensional model representing the dentition of a patient. The virtual three-dimensional model can be obtained from one of several possible sources; in the preferred embodiment it is arrived at from a scanning of the dentition. The apparatus further includes software executable by the processing unit that accesses the model and displays the model on the display of the workstation. The software further includes navigation tools, e.g., typed commands, icons and/or graphical devices superimposed on the displayed model, that enables a user to manipulate the model on the display and simulate the movement of at least one tooth in the model relative to other teeth in the model in three-dimensional space, and quantify the amount of movement precisely. This simulation can be used, for example, to design a particular target situation for the patient.
The development of a unique target situation for the patient has utility in a variety of different orthodontic appliances, including an approach based on off-the-shelf or generic brackets and a custom orthodontic archwire. The scope of the invention is sufficient to encompass other types of appliances, such as an approach based on customized brackets, retainers, or removable aligning devices. In a bracket embodiment, the memory contains a library of virtual, three-dimensional orthodontic brackets. The software permits a user to access the virtual brackets through a suitable screen display, and place the virtual brackets on the virtual model of the dentition of the patient. This bracket bonding position can be customized on a tooth by tooth basis to suit individual patient anatomy. Because the tooth models, brackets and archwire are individual objects, and stored as such in memory, the treatment planning apparatus can simultaneously display the virtual brackets, the archwire and the virtual model of the dentition, or some lesser combination, such as just the brackets, just the dentition, or the brackets and the archwire but not the teeth. The same holds true with other appliance systems.
In a preferred embodiment, the virtual model of teeth comprises a set of virtual, individual three-dimensional tooth objects. A method of obtaining the tooth objects from a scan of teeth, and obtaining other virtual objects of associated anatomical structures, e.g., gums, roots and bone is described. When the teeth are separated from each other and from the gums, they can be individually manipulated. Thus, the individual tooth objects can be individually selected and moved relative to other teeth in the set of virtual tooth objects. This feature permits individual, customized tooth positioning on a tooth by tooth basis. These positioning can be in terms or angular rotation about three axis, or translation in transverse, sagittal or coronal planes. Additionally, various measurement features are provided for quantifying the amount of movement.
One of the primary tools in the treatment planning apparatus is the selection and customization or a desired or target archform. Again, because the teeth are individual tooth objects, they can be moved independently of each other to define an ideal arch. This development of the target archform could be calculated using interpolation or cubic spline algorithms. Alternatively, it can be customized by the user specifying a type of archform (e.g, Roth), and the tooth are moved onto that archform or some modification of that archform. The archform can be shaped to meet the anatomical constraints of the patient. After the initial archform is designed, the user can again position the teeth on the archform as they deem appropriate on a tooth by tooth basis. The treatment planning software thus enables the movement of the virtual tooth objects onto an archform which may represent, at least in part, a proposed treatment objective for the patient.
Numerous other features are possible with the treatment planning software, including movement of the teeth with respect to the other teeth in the archform, changing the position of the virtual brackets and the teeth with respect to each other, or opposing teeth with respect ot the selected archform. Custom archwire bends can be simulated to provide additional corrections. Bonding corrections at the bracket-tooth interface are also possible.
In another aspect of the invention, a method is provided for digital treatment planning for an orthodontic patient on a workstation having a processing unit, a user interface including a display and software executable by the processing unit. The method comprises the steps of obtaining and storing a three-dimensional virtual model of teeth representing the dentition of the patient in a current or observed situation. The virtual model is displayed on the display. The method further includes the step of moving the position of teeth in the virtual model relative to each other so as to place the teeth of the virtual model into a target situation and displaying the virtual model with the teeth moved to the target situation to the user. Parameters for an orthodontic appliance to move the patient""s teeth from the current situation to the target situation can be derived from the virtual model and the target situation. For example, if virtual brackets are placed on the teeth, their location in the target situation can dictate the design of an archwire to move the teeth to the target situation.
The scanner system is provided for capturing three-dimensional information of an object. The object can be virtually any object under scrutiny, however the present document will describe an application in which the object is the dentition of a patient suffering from a malocclusion.
The scanning system enables three-dimensional surface information to be obtained with a very high decree of precision. Moreover, the scanning system can be used without requiring precise movement of the scanner, or requiring the object under scrutiny to be fixed in space. Surprisingly, the scanner is able to generate precise three dimensional surface information by simply moving the scanner over the surface of the object, such as by hand, in any manner that is convenient for the user, even if the object moves in any random direction during the scanning within reasonable limits. Thus, the scanner can be used to capture the surface of a patient""s dentition in a minute or two, even if the patient moves their head or jaw while the scanning is occurring. Precise knowledge of the spatial relationship between the scanner and the object is not required.
The scanner obtains a set of images, which are processed in a computer to calculate the surface configuration of the object in three dimensions of space automatically, quickly, with high precision, and with essentially no human involvement other than the act of scanning. The precision or accuracy will be dictated largely by the extent to which the object under scrutiny tends to have undercut or shadowed features which are difficult to detect, necessitating a narrow angle between the projection and imaging axes. For teeth, an accuracy of under 20 or 30 microns is possible. This accuracy can be further improved depending on the nature of the surface, such as if the surface does not have a lot of undercut or shadowed features, by increasing the angular separation of the projection axis and the imaging axis.
Each image captured by the scanner is converted to a virtual, three-dimensional point cloud or xe2x80x9cframe.xe2x80x9d The illustrated embodiment has a relatively coarse resolution for any single frame, due to a coarse projection pattern, but a fine resolution is obtained by obtaining multiple images and performing a registration procedure on the frames, as described below. Since each point on the surface of the object is captured in a plurality of images (such as five or six in a typical example of scanning), the registration of frames results in a fine resolution. An even finer resolution can be obtained by scanning slower and capturing more images of the surface of the object from different perspectives and registering the resulting frames to each other.
This surface configuration of the object in three dimensions of space can be represented as a mathematical model, i.e., a virtual model of the object, which can be displayed on any workstation or computer using available software tools. The mathematical model can be viewed in any orientation in space, permitting detailed analysis of the surface. The model can be compared to template objects stored in a computer. Deviations in the object from the template can be quantified and analyzed. Further, the virtual model can be transported from one computer and another computer anywhere in the world essentially instantaneously over communications links such as the Internet. The model can be replicated in a computer and thus shared and used by multiple users simultaneously.
The scanner system is useful for a wide variety of industrial, medical, archeological, forensic, archival, or other purposes. Furthermore, the scanner can be scaled down in size such that it can be hand-held and used to scan small objects, e.g., teeth or small machined parts, or scaled up in size so that it can be used to make mathematical models of larger scale objects such as works of art, sculptures, archeological sites (e.g., the caves at Lascaux, France or the dwellings or kivas in Mesa Verde National Park), rooms or building facades.
In accordance with a preferred embodiment, the scanner system includes a projection system that projects a pattern onto the object along a first optical axis. The pattern may consist of parallel lines, parallel lines separated by shapes or colors, such as colored dots, or other suitable pattern. The projected pattern is used to gather information as to the surface characteristics of the object in accordance with the methods and procedures described in more detail below.
The scanner further includes an electronic imaging device, preferably in the form of a charge-coupled device comprising a two-dimensional array of photo-sensitive pixels. The electronic imaging device is oriented along a second optical axis different from the first optical axis. The electronic imaging device forms an image of the pattern after reflection of the pattern off of the object under scrutiny. The surface configuration of the object will cause the projection pattern to become distorted and changed, and thereby provide information as to the surface configuration. This information as to the surface is captured by the imaging device as two-dimensional images of the reflection pattern. These images are processed in accordance with procedures described herein to derive three-dimensional information as to the surface of the object.
The scanning system, in a preferred embodiment, further includes a memory that stores data representing a three axis (X, Y, Z) calibration relationship for the scanner. The calibration relationship can be in the form of a table or in the form of a mathematical function (e.g., polynomial, spline, or other function). The calibration relationship identifies two properties of the scanner: (1) pixel coordinates for the electronic imaging device for numerous portions of the pattern, said pixel coordinates associated with distance information from the projection system in a Z direction at at least two different Z distances, and (2) distance information in X and Y directions, for the numerous portions of said pattern, at the at least two different Z distances. A method of obtaining the calibration relationship is described in detail below. While the simplest form of the relationship is a table, as described in detail below, these calibration relationships could be equivalently represented by one or more mathematical functions as will be apparent to those skilled in art.
The calibration relationship is used to derive three-dimensional coordinates for points on the surface of an object imaged by the imaging device. The generation and use of the preferred calibration table is also explained in further detail below. Other calibration tables or procedures are also possible. The use of the calibration relationship allows the scanner to operate without precise knowledge of the optical and mechanical properties of the scanner, as is required in prior art systems.
The scanning system further includes at least one data processing unit, e.g., the central processing unit of a computer or a digital signal processor, which processes the images of the projection pattern after reflection from the surface of the object. The processing unit compares data from the image (e.g., pixel locations where certain points in the projection pattern are imaged) to the calibration relationship to thereby derive spatial information, in three dimensions, of points on the object reflecting the projected pattern onto the electronic imaging device. Multiple processing units can be used to reduce the amount of time it takes to process the two-dimensional images, calculate three-dimensional coordinates for points in each image, and register frames of three-dimensional coordinates relative to each other to generate a complete virtual model of the object.
The scanning system thus derives three-dimensional information of the object from the images generated by the imaging device and from the calibration relationship stored in memory. Precise knowledge of the optical characteristics of the scanner, the angles between the first and second optical axes, the angle between the optical axis and the point in the imaging device, or the separation distance of the projection device from the imaging device are not necessary. The projection system and the imaging device can be even uncoupled relative to each other. The calibration relationship automatically and completely compensates for these issues, as well as manufacturing variations and tolerances in the scanner optics, in a highly reliable manner. Moreover, knowledge of the distance from the scanner to the object is not required. Additionally, control over the distance between the scanner and the object is not required, within reasonable limits dictated by the depth of focus of the imaging optics. Ideally, during scanning the distance from the scanner to the object is maintained within the limits of the Z distances used during calibration, and that distances is within the combined depth of focus of the projection optics and the imaging optics.
The scanning system can be constructed such that the memory, processing unit, and optical elements are in a single unit. Alternatively, the processing unit and memory can be located at a separate location, such as a scanning workstation or xe2x80x9cscanning nodexe2x80x9d, in order to reduce the size of the scanner per se. In such an embodiment, the projection system and the image-recording device can be miniaturized into a hand-held scanner device. A suitable cable connects the scanner device to the workstation to thereby supply the processing unit with scan data, and to receive commands (illumination commands, start/stop commands, etc.) from the workstation.
Preferred appliance manufacturing systems are also described, including bending apparatus that bends medical devices, such as orthodontic appliances like archwires, fixation plates and other devices. Bracket placement trays are also described. As one embodiment, an orthodontic appliance manufacturing system is provided comprising a machine readable memory storing digital data representing a three-dimensional virtual model of a malocclusion of a patient and digital information representing the location of orthodontic brackets to be placed on the malocclusion, said memory further storing digital data representing a three-dimensional virtual model of the patient""s teeth and the dentition and location of orthodontic brackets at a target situation in three dimensions. The apparatus further includes a wire bending robot. A rapid prototyping instrument such as SLA (stereolithography) is provided for generating a three-dimensional physical model of the malocclusion with the brackets placed on the malocclusion. A forming device forms a bracket placement tray matching the geometry of the physical model of the malocclusion with the brackets. The bracket placement tray may be removed from the physical model, said tray formed with spaces for placement of brackets and enabling bonding of the brackets to the teeth at a desired location based on the virtual model of the dentition and the placement of orthodontic brackets on the malocclusion.
Numerous other features of the appliance manufacturing apparatus, the treatment planning software, the scanning system and related features will be more apparent from the following detailed description.